| |
|
|
|
Robots.txt is a visual editor for Robot Exclusion Files and a log analyzer software. It allows a user to quickly and easily create the robots.txt files required to instruct search engine spiders, which parts of a Web site are not to be indexed and made searchable by the general Web public and then to identify spiders, which do not keep to those instructions.
The program provides the user a way to log onto his FTP or local network server and then select the documents and directories which are not to be made searchable.
By means of this program you will be able to visually generate industry standard robots.txt files; identify malicious and unwanted spiders and ban them from your site; direct search engine crawlers to the appropriate pages for multilingual sites; use robots.txt files for doorway page management; keep spiders out of sensitive and private areas of your Web site; upload the correctly formatted robots.txt files directly to your FTP server not switching from Robots.txt Editor; track spider visits; create spider visits reports in HTML, Microsoft Excel CSV and XML formats and more. Free program`s updates and upgrades unrestricted in time; unlimited number of Web sites to work with |
| « read more about Robots.txt 2.0 |
|
|
RECOMMENDED DOWNLOADS IN THIS CATEGORY
|
|
|
|
| » Site Map Pro 2.2
|
| Site Map Pro - Create a site map or Google Sitemap Protocol for any web site. Site Map Pro will examine a web site, mapping all the pages that it finds. It will then create a single HTML site map file. By simply placing the single HTML file on your web s ... |
|
| Site Map Pro | 1943KB | Win 95/98/ME/NT/2000/XP | $59 | Downloads: 1130 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » Web Optimization Angel 2005.6.2
|
| Web Optimization Angel :Top 10 website search engine optimization tool Easy to improve your site`s rank by optimizing the most important ranking factors. Every keyword or keyword phase will generate 45 search engine ranking factors: nine important rank ... |
|
| Web Site Promotion Software | 930KB | Win 2000/XP | $69.95 | Downloads: 1145 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » Notepad Pro 2.9
|
| Professional Notepad is a powerful editor that allows you view and edit HTML, CSS, JavaScript, PHP, PERL, SQL, Delphi, C++ and other languages source code. It is a advanced tool supporting the features you`ve always dreamt about, such as syntax highlight ... |
|
| A Tech Group | 400KB | Win 95/98/ME/NT/2000/XP | $18 | Downloads: 1183 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » CSE HTML Validator Professional 7.00
|
| CSE HTML Validator Professional is a fast, powerful, highly user configurable, and easy to use professional web development tool that helps you create syntactically correct and accessible HTML, XHTML, and CSS documents. The new HTML Tidy Tool can even fix ... |
|
| AI INTERNET SOLUTIONS | 4386KB | Win 98/ME/NT/2000/XP | $129 | Downloads: 1205 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » Blueframe Web 3.0.0
|
| For individuals and companies wanting to quickly create a professional website presence. Blueframe Web is a unique technology which gives you 30 Flash, PocketPC website templates, plus an easy-to-use WYSIWYG authoring software. No complicated programming ... |
|
| Framewood Technologies | 5712KB | Win 98/ME/NT/2000/XP | $99.95 | Downloads: 1250 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » AlbumAdvanced 50620.1458
|
| This AlbumAdvanced Photo Album looks through your web server in the directories in which you specify for both gif and jpeg images. It displays them as thumbs, as well as showing all child directories. As the user browses these directories, they can view t ... |
|
| CrookedBush.com Inc. | 25.42KB | Linux | $4.68 | Downloads: 1147 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » Flash Website Templates - Auora Website 4.03
|
| No need to be a computer expert or buy expensive website software to create your own Flash website. With Auora Website you get 4 Flash website templates with 3 themes each, that is 12 different styles! plus you get an `easy-to-use` software where you can ... |
|
| Framewood Technologies | 5673KB | Win 98/ME/NT/2000/XP | $79.95 | Downloads: 1168 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » Webmaster Toolkit 1.0
|
| Webmaster Toolkit includes many professional script authoring tools into one program. Many of them are winning-award products. With this easy-to-use web authoring toolkit, you can easy to draft, design, develop and deploy scripts such as menu, navigation, ... |
|
| USINGIT.COM | 8878KB | Win 95/98/ME/NT/2000/XP | $99.95 | Downloads: 1162 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » Professional Notepad 2.91
|
| Professional Notepad is a powerful editor that allows you view and edit HTML, CSS, JavaScript, PHP, PERL, SQL, Delphi, C++ and other languages source code. It is a advanced tool supporting the features you`ve always dreamt about, such as syntax highlight ... |
|
| A Tech Group | 404KB | Win 95/98/ME/NT/2000/XP | $18 | Downloads: 1135 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » Website Templates - Blueframe Web Enterprise Pack 3.0.0
|
| For Hosting Companies, ISPs, and Developers or similar. For as little as 19¢ per customer you now have a product which will help you do so. Blueframe Web is a unique technology which gives your customers 30 Flash, PocketPC website templates, plus an eas ... |
|
| Framewood Technologies | 5712KB | Win 98/ME/NT/2000/XP | $599.95 | Downloads: 1127 |
| Date added: Sep 23, 2005 |
» Details |
|
|
|
|
| » Web Weaver Gold 2005
|
| Create exciting Web pages quickly and easily with this feature-rich HTML editor! Tutorials and wizards step you through Web page creation making you an HTML expert in no time! It features colored HTML code for easier editing; wizards for creating advance ... |
|
| McWeb Software, Inc. | 8402KB | Win 95/98/ME/NT/2000/XP | $34.95 | Downloads: 1146 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » KVT ImageCutter 1.5
|
| This programm is intended for all WWW authors. It cuts large graphic images in few pieces and then creates the HTML file with the table which shows an original picture. Then you can add links to pieces to create some kind of an image map, but even if you ... |
|
| KVT Software | 1108KB | Win 95/98/ME/NT/2000/XP | $14.95 | Downloads: 1232 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » w3 I.D.E. NexGen Edition 2.0.2
|
| Build and publish your website online with ease using w3IDE. From small personal sites to huge corporate outlets, w3IDE is designed to meet the needs of both novice and expert users. Supports HTML, XHTML, PHP, Perl (cgi), XML, JavaScript, CSS, Java and ... |
|
| Bannon Productions | 7358KB | Win | $25.00 | Downloads: 1184 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » phpAdBoard 1.8
|
| phpAdBoard includes everything needed to run your own High Quality, maintenance free market-place on the Internet. Imagine the success you will have with Standard features that include Easy installation with a user-friendly installation Wizard; phpAdBoard ... |
|
| W2B Web to Business | 55KB | Win 98/ME/NT/2000/XP | $39.95 | Downloads: 1229 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » LinkMachine 1.25
|
| The key to success for a website or online business is good search engine ranking. All of the major search engines use link popularity to determine how well a site will rank - so the more high quality, relevant links that lead to your site, the higher it ... |
|
| IronZog | 69KB | Win 3.x/95/98/ME/NT/2000/XP | $0.00 | Downloads: 1248 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » DatingClub 5.0
|
| Dating Club is a powerful and complete web application that meets today`s needs for a specialized dating website. Dating Club assures user confidentiality while providing features such as matchmaking, complete horoscope, message inbox, favorites list, ema ... |
|
| W2B Web to Business | 540KB | Win ME/NT/2000/XP | $299.95 | Downloads: 1186 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » phpWebDirectoryPro 1.02
|
| phpWebDirectory Pro is a full-featured Web Directory Application, which includes an integrated User Management system and Easy Installation with the help of user-friendly installation wizard and everything needed to create your own fully customizable, Int ... |
|
| W2B Web to Business | 70KB | Win 98/ME/2000/XP | $74.95 | Downloads: 1124 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » QS Flash Magic Menu Builder 1.1
|
| This component has been designed for use with Flash MX 2004 and Flash MX Professional 2004. Documents and menu-editing software included!. We are proud to introduce a new menu-building tool. This product provides tremendous flexibility by offering a wid ... |
|
| Quadricsoft Inc. | 4003KB | Win 98/2000/XP | $30 | Downloads: 1127 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » Applet Glide Navigation Pro 2005 1.0
|
| The applet can be customized highly and easily through the ACP(Applet Configuration Program). You can specify caption, image, font, color, link and more. It is unnecessary to write any code manually. ACP(Applet Configuration Program) will generate all nec ... |
|
| USINGIT.COM | 823KB | Win 95/98/ME/NT/2000/XP | $49.95 | Downloads: 1145 |
| Date added: Sep 23, 2005 |
» Details |
|
|
| » AxpDataGrid WebControl for ASP.NET 2.8.6
|
| ASP.Net WebControl for SQL queries and .Net DataSources. Automatic paging and sorting. Scrollable grid for major browsers.Form Edit. Rich Text Editor. Copy Functionality. 35 skins ready to use. MSSQL,Access,Oracle etc. 23 Samples for C# and VB. Visual Stu ... |
|
| Axezz | 2217KB | Win 2000/XP | $149 | Downloads: 1275 |
| Date added: Sep 23, 2005 |
» Details |
|
|
|
|
| |
